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Abstract. We consider a class of multivariate recurrences frequently arising in the worst case analysis of Davis- 
Putnam-style exponential time backtracking algorithms for NP-hard problems. We describe a technique for proving 
asymptotic upper bounds on these recurrences, by using a suitable weight function to reduce the problem to that of 
solving univariate linear recurrences; show how to use quasiconvex programming to determine the weight function 
yielding the smallest upper bound; and prove that the resulting upper bounds are within a polynomial factor of 
the true asymptotics of the recurrence. We develop and implement a multiple-gradient descent algorithm for the 
resulting quasiconvex programs, using a real-number arithmetic package for guaranteed accuracy of the computed 
worst case time bounds. 

1 Introduction 

The topic of exponential-time exact algorithms for hard problems has led to much research in recent years 
EQIHlfroiEIElEin^lEl- In contrast to the situation with polynomial time algorithms, one can not 
significantly increase the size of a solvable instance by waiting for Moore's law to provide faster computers, 
so algorithmic improvement is especially important in this area. Several design principles are known for such 
algorithms, including dynamic programming 1 17 1 and randomized hill climbing 1 19J, but the most common 
approach is a simple form of branch-and-bound in which one repeatedly performs some case analysis to find 
an appropriate structure in the problem instance, and then uses that structure to split the problem into several 
smaller subproblems which are solved by recursive calls to the algorithm. 

As an example of this approach, a graph coloring algorithm of the author UJJ uses a subroutine for 
listing each maximal independent set of at most k vertices (/:-MIS) in an n-vertex graph. This subroutine 
(slightly simplified for this example) repeatedly selects and applies one of the following cases: 

- If the input graph G contains a vertex v of degree zero, recursively list each {k — 1)-MIS in G \ {v} and 
append v to each listed set. 

- If the input graph G contains a vertex v of degree one, with neighbor u, recursively list each {k — 1)-MIS 
in G \ N{u) and append u to each listed set. Then, recursively list each {k — 1)-MIS in G \ {u,v} and 
append v to each listed set. 

- If the input graph G contains a path V1-V2-V3 of degree-two vertices, then, first, recursively list each 
{k - 1)-MIS in G \ A'^(vi ) and append vi to each hsted set. Second, hst each {k - 1)-MIS in G \ A'^(v2) 
and append V2 to each listed set. Finally, list each {k — 1)-MIS in G \ {{v\} VJN{v^)) and append V3 to 
each listed set. Note that, in the last recursive call, vi may belong to A'^(v3) in which case the number of 
vertices is only reduced by three. 

- If the input graph G contains a vertex v of degree three or more, recursively list each fc-MIS in G \ {v}. 
Then, recursively list each {k — 1)-MIS in G \ A'^(v) and append v to each listed set. 

It is not hard to see that any graph contains at least one of these cases. We can bound the worst-case number 
of output sets, as a recurrence in the variables n and k: 




As base cases, T(0, 0) = 1, T{n, —1) =0, and T{n, k) = for ^ > n. Each term in the overall maximization 
of the recurrence comes from a case in the case analysis; the recurrence uses the maximum of these terms 
because, in a worst-case analysis, the algorithm has no control over which case will arise. Each summand 
in each term comes from a recursive subproblem called for that case. It turns out that, for the range of 
parameters of interest n/4 < k < n/3, the recurrence above is dominated by its last two terms, and has 
the solution T{n,k) = (4/3)" (3^^/43)'^. We can also find graphs having this many /c-MISs, so the analysis 
given by the recurrence is tight. Similar but somewhat more complicated multivariate recurrences have 
arisen in our algorithm for 3-coloring 1 10 1 with variables counting 3- and 4- value variables in a constraint 
satisfaction instance, and in our algorithm for the traveling salesman problem in cubic graphs |12| with 
variables counting vertices, unforced edges, forced edges, and 4-cycles of unforced edges. 

These examples of recurrences have all had few enough terms that they can be solved by hand, but 
Table ^depicts a recurrence, arising from unpublished work with J. Byskov on graph coloring algorithms, 
that is complex enough that hand solution seems unlikely. This recurrence was derived through an iterative 
process, starting from a simple case analysis of the problem, in which the worst cases for the algorithm were 
repeatedly identified and replaced by a larger number of better cases. 

Our interest in this paper is in performing this type of analysis algorithmically: if we are given as input 
a recurrence such as the ones discussed above, can we efficiently determine its asymptotic solution, and 
determine which of the cases in the analysis are the critical ones for the performance of the backtracking 
algorithm that generated the recurrence? We show that the answer is yes, by expressing the problem as a 
quasicomex program, a type of generalized linear program studied previously by the author and others with 
applications including finite element mesh smoothing |2|, brain fiat mapping, hyperbolic graph drawing, 
and conformal meshing |4|, and multi-projector tiled display color gamut equalization |5|. This quasicon- 
vex programming formulation allows us to solve a r-term recurrence in 0{t) steps, each of which involves 
the solution of a constant number of algebraic equations; alternatively, we can apply any of several numer- 
ical hill-climbing techniques, which are guaranteed to converge to the global optimum of a quasiconvex 
program. We describe one such technique and provide two proof-of-concept implementations of it, one for 
exploratory analysis using floating point, and another using the XR exact real-number computation package 
for Python fT8l l6ll. The algorithms we describe are able to analyze recurrences such as the one in Tabled 
producing asymptotic analysis of their behavior (here, we are interested in the asymptotics of r(?i,0)) and 
identifying the cases that are bottlenecks for that analysis, typically within one or two seconds for a floating 
point evaluation sufficiently accurate for exploratory analysis of algorithms. 

We first began using a weighting technique for upper bounding recurrences of this type in our previous 
work on graph coloring |10|, without much regard to its completeness or generality until this paper. De- 
spite some searching we have been unable to find relevant prior research on asymptotic analysis of similar 
multidimensional recurrences. 

2 Formalization and Statement of Results 

We assume that the input to our problem consists of the following items: 

- An integer dimension d. 

- A recurrence 

F{x) = max Vf(x - 5ij), 

where x and Sij are vectors in 7/, and / and 7 are indices ranging over the cases and subproblems of the 
algorithm to be analyzed. We assume that, as base cases for the recurrence, F(0) = 1, and F{y) = 
when y can not reach the zero vector by any expansion of the recurrence. 

- A target vector t'ml/. 



f T(n + 3, h - 2) + T(n + 3, h - I) + T{n + 4, h - 2) + T(n + 5, h - 2), 
T(n,h+1) + T(n+l,h + 2), 
lT{n + 2, h) + 2T(n + i,h), 

2 T(n + 2, A) + 2 r(n + 3, A) , 

T(n + 3,h - 2) + T(n + 3,h - I) + T(n + 5,h - 3) + T(n + 5,h - 2), 
T(n + LA) + T(n + 3,1, - 1) + 3 T(n + 3, h + 3), 
T[n + 3,/! - 2) + 2 r(n + 3, A - 1) + r(n + 7, A - 2), 
r(;i + 1, A) + 2r(n + 4, ft - 2), 

3 r(;i + 1, A + 2) + 2r(n + 1, A + 5), 

2T(n + 2, A) + T{n + 3,h+\) + T(n + 4, A) + T{n + 4, A + 1), 

T{n + 1,A - 1) + r(n + 4,A - 1), 

r(ii + 1,A + 3) + 2T(n + 2,A) + T(n + 3,h), 

2T(n + 2,h- 1), 

T(n, h + 3) + T(n+l,h + 2) + T{n + 2,h), 
T(n + 1, A - 1) + T(n + 4, A - 1), 

2r(fi + 1, A + 1) + r(n + 2, A + 1), 

9T(n + 2, A + 3), 

r(n + 1, A) + r(n + 1, A + 1), 

9 ^(n + 9, A - 5) + 9 ^(n + 9, A - 4) , 

T(n + 3,h - 2) + T(n + 3,h- 1) + T(n + 5,h- 2) +2T(ji + 6,h- 3), 

T(n + 1,A - 1) + r(n + 4,A) + T(n + 4, A + 1), 

2T(ii + 2, A) + r(n + 3, A) + r(n + 4, A) + r(B + 5, ft), 

T{n + 1, A) + 2r(n + 2, A + 1), 
T(n + 1,A - 1), 

2r(n + 2, A + 1) + r(n + 3, A - 2) + r(n + 3, A), 
^(i! + 1, A + 1) + r(n + 1, /i + 2) + r(n + 2, A), 
2T(n + 2, A) + 2r(n + 3, /i), 

r(n + 1,A + 2) + r(n + 2,A - 1) + r(n + 2,A + 1), 
r(n+ 1,A), 

r(n + 2, A + 1) + r(n + 3, A - 2) + r(i> + 4, ft - 3), 
r(n - 1, A + 2), 

3 r(n + 4, ft) + 7 ^(n + 4, A + 1), 
r(n + 2,ft - 1) + 2T(n + 3,ft - 1), 

T(n + 2, A - 1) + r(n + 2, A) + r(/i + 2, A + 1), 
r(n + 3, A - 2) + r(ii + 3, A) + 2 r(n + 4, A - 2) , 

r(ii + 1, A) + r(ii + 3, A - 1) + r(ii + 3, A + 3) + r(ix + 5, A) + r(/i + 6, A - 1), 

2 r(n + 1, ft + 4) + 3 r(ii + 3, A + 1) + 3 r(n + 3, A + 2), 

3r(i! + 3,ft + 1) + r(ii + 3,A + 2) + 3r(n + 3, A + 3) +3 7'(n + 4,ft), 

r(n + 2, A - 1) + r(ii + 3, A - 1) + r(n + 4, A - 2), 

r(;i,A + 1), 

r{;i + 1,A + 2) + T(n + 3, A - 2) + r(n + 3, A - 1), 

2r(n + 3, A - 1) + r(ii + 3, A + 2) + r{n + 5, A - 2) + r(fi + 5, A - 1) + r(n + 5, A) + 2r(n +7, ft - 3), 
T(n + 2, A + 2) + 2r(ii + 3, A) + 3T(n + 3, A + 1) + T(n + 4, A), 

T(n + 3, A - 2) + r(/i + 3, A - 1) + r(n + 5, A - 3) + r(n + 6, A - 3) + r(n + 7, A - 4), 

r{;i + 1,A - 1), 

r{;i + 1, A) + 2T(n + 3, A), 

4r(// + 3, A + 1) + 5 r(n + 3, A + 2), 

4r(ii + 2, A + 3) + 3 r(n + 4, A) + 3 r(n + 4, A + 1), 

r(n + 3, A - 2) + 2r(n + 3, A - 1) + r(n + 6, A - 3), 

4 7'(n + 2, A + 3) + 6 r(n + 3, A + 2) , 
T(n,h+ 1) + r(n + 4, A - 3), 

T(n + 1, A - 1) + 2r(n + 3, A + 2), 

2r(ii + 2,A+ 1) +3r(ii + 2,A + 3) + 27'(n + 2, A + 4), 
2T(n + 2,h) + 2T(n + 2,h + 3), 

2 r(n + 2, A) + r(n + 2, A + 3) + r(n + 3, A + 2) + + 4, ft) + ^(n + 4, A + 1), 
2r(n,A + 2), 

r(n + 2, A) + r(n + 3, A - 2) + r()i + 3, A - 1), 

r(;i + 3, A - 2) + 2r(n + 4, A - 2) + r(n + 5, A - 3), 

r(n + 1,A) + r(ii + 5, A - 4) + r(n + 5,A - 3), 

T(n + 1,A + 2) + r(n + 2,A - 1) + r(B + 3, ft - 1), 

r(n + 2,A- 1) +r(n + 2,A) + 7'(n + 4,ft- 1), 

10r(n + 3,A + 2), 

6r(n + 2, ft + 2), 

r(n + 2,ft) +T(n+3,h), 

2r(ii + 3, A - 1) + T(n + 3,h + 2) + r(B + 5,ft - 2) + r(ii + 5,ft - 1) + T(n + 5,h) + T(n + 6,h- 2) +7'(n + 7,A - 2), 

6r(n + 3, A + 1), 

3 T(n,h + 3), 

r(n + 2, A - 1 ) + r(n + 2, A) + T(n + 4, A - 2) , 
2T{n + 5, A - 3) + 5r(n + 5, A - 2), 
2r(n + 2, A) + r(n + 2, A + 1) + T(n + 4, A - 1), 
8 r{;i + 1, A + 4), 

r(n + 3, A - 2) + r(n + 3, A - 1) + r(ii + 5, A - 3) + T{n + 5, A - 2) + r(n + 7, A - 3), 
r(B + 1, ft - 1) + r(n + 2, A + 2), 

5 r(n + 2, ft + 2) + 2 ^(n + 2, ft + 3) 



Table 1. A recurrence arising from unpublished work with J. Byskov on graph coloring algorithms. 



smallmis 

"degO" : 
"degl" : 
"deg2" : 
"deg3" : 

} 



{ 

[ (1,1) ] , 
2*[ (2,1) ] , 
3*[ (3,1) ] , 

[(4,1), (1,0) 



Table 2. Python representation of recurrence input. 



The desired output is a description of the asymptotic behavior of the function /(«) = F{nt). We call the 
expressions ~ ^ij) terms of the recurrence, and their subexpressions F{x — djj) summands of the 

term. Much of what we describe here would generalize without difficulty to non-integer values of x and (5, j, 
and to non-integer multipliers on the summands of each term. 

For our Python implementation, we represent the recurrence as a dictionary mapping case names to 
terms, with each term represented as a list of summands and each summand represented as a <i-tuple of 
integers. For instance, the cardinality-bounded maximal independent set recurrence discussed in the intro- 
duction has the representation shown in Table |2l This Python representation also allows the inclusion of 
comments in the file containing the recurrence, describing in further detail the case analysis from which the 
recurrence arises. 

We obtain the following results: 

- We show that our recurrences can be upper bounded by linear univariate recurrences formed by weight- 
ing the variables, and that the optimal set of weights for this upper bound technique can be found 
efficiently using quasiconvex programming. The optimal basis for the quasiconvex program consists of 
the terms forming the worst cases for the recurrence. 

- We describe a numerical improvement technique for quasiconvex programming, based on multi-gradient 
descent, and provide two implementations of this technique: one based on floating point arithmetic and 
capable of running at interactive speeds for exploratory algorithm analysis, and one using an exact 
arithmetic package for publishable guaranteed worst case bounds. 

- We prove lower bounds showing that the upper bounds from our optimal weighting technique are tight 
to within a polynomial factor. 

3 Upper Bounds 

In this section we describe a weighting technique that can be used to obtain asymptotic upper bounds for 
our recurrences, and formulate the problem of optimizing the weights in order to obtain the best such upper 
bound. 

Fix a vector w G M^, such that, for each summand F{x — djj) of the input recurrence, w ■ djj is positive, 
let y be a real variable, and define 

F„{y) = max F{x). 

w-x<y 

Then it is not hard to see, by using the recurrence for F{x) to expand the right hand side of this definition, 
and interchanging the order in the maximization, that 

Fwiy) < max^ F,^{y - w ■ 5ij). 
j 

As base cases set F^iy) = 1 for < y < min,jW • 6ij and F^,(y) = for y < 0. This resembles the 
recurrence defining F, but with a real instead of vector-valued argument. For linear univariate recurrences 



similar to this one, standard solution techniques such as generating functions and characteristic polynomials 
are commonly taught in freshman combinatorics courses. The recurrence for F„ is nonlinear because of the 
maximization, and involves non-integer variables, but the same techniques apply: 

Lemma 1. F„(y) = O(c^), where is the unique positive root of the monotonic function 

r^{c) = 1 -max^c-"''^'V. 

j 

It will be convenient later to separate out this analysis into the different terms of the recurrence: 

Lemma 2. Let c^j denote the unique positive root of the monotonic function r„^i{c) = 1 — c~^'^'j. Then 
— max^' c^fj. 

Since was defined as a maximum of values of F, this technique immediately leads to upper bounds 
for the target function/(?i) = F{n t): 

Lemma 3. Let w G be such that, for each summandF{x — 6ij) of the input recurrence, w ■ 6ij is positive, 
and let w ■ t = 1. Thenf{n) < F„{n) = 0{c"^). 

We call w a weight vector, because if one interprets the coordinates of w as weights of the different 
backtracking algorithm instance features counted by x, then w • x is the total weight of the instance. The 
recurrence for F^ and its solution 0{c"^,) describes the time for the backtracking algorithm as a function of 
instance weight. However, different choices of the weight vector will give different upper bounds 0{d^) on 
the time used by the backtracking algorithm. Our task now is to select the best weight vector, that is, the one 
yielding the tightest upper bound. For convenience, we define Cw = c^^i = +oo when w ■ 6jj is non-positive 
for some j. 

4 Quasiconvex Programming 

In this section we find efficient algorithmic solutions for the optimal weighting problem formulated in the 
previous section. 

A function q{w) : i-^ M is called quasiconvex when its level sets q-'^ = {w G | q{x) < A} 
are all convex. In particular, the points w where q achieves its minimum value (if a minimum exists) form 
a convex set, and an approximation to a value achieving the global minimum can be found numerically 
by local improvement techniques. If the functions qj for / in some finite index set S are all quasiconvex, 
then the function qs{w) = max,g5 <7,(w) is also quasiconvex, and it becomes of interest to find a point 
where qg achieves its minimum value. Amenta et al. 1 2 1 define quasiconvex programming as a formalization 
of this search for the minimum of qg. Linear programming can be seen as a special case of quasiconvex 
programming in which all the functions qj are linear. 

More formally. Amenta et al. define a nested convex family to be a map k{X) from the nonnegative real 
numbers to compact convex sets in M.'^ such that if Ai < A2 then k{Xi) C k(A2), and such that for all A, 
k{X) = riA'>A k{X'). Any nested convex family k determines a quasiconvex function q^iw) = inf { A [ w G 
k(A) } on M^, with level sets q^'^ = k{X). Conversely, when q is quasiconvex, the closures of the level sets 
q-^, restricted to a compact convex subdomain of M^, form a nested convex family. 

Amenta et al. define a quasiconvex program to be formed by a set of nested convex families S = 
{ki, K2, . ■ ■ K„}; the task to be solved is finding the value 

A*(5) =inf|(A,w) \ we f] Ki{X)} 



where the infimum is taken in the lexicographic ordering, first by A and then by the coordinates of w. 

Quasiconvex programs can themselves be seen as a special case of generalized linear programs. These 
are optimization problems based on an objective function that maps sets to totally ordered values and that 
satisfies certain axioms fll lTBllTSl ; quasiconvex programs are generalized linear programs because the func- 
tion A* {S) defined above satisfies these axioms 1 2 1 . One consequence of this generalized linear programming 
formulation is that the value of any quasiconvex program is determined by a basis, a subset of the nested 
convex families with cardinality 0{d). Another consequence is that, when the dimension d is bounded, 
quasiconvex programs can be solved by dual-simplex based randomized algorithms that perform a number 
of steps linear in the number of nested convex families in the input, where each step consists of solving a 
constant-sized subproblem. 

As we now show, our problem of finding an optimal weight vector for our input recurrence can be 
expressed in this quasiconvex programming framework. 

Lemma 4. Let c^^i be as defined in the previous section. Then the function qi{w) = c„j is quasiconvex. 

Proof. We must show that qf^ = {w \ Cw,i < A} is convex. Equivalently, expanding the definition of c^^t 
and using the monotonicity of the function rn, we must show convexity of the set 

= I r„,iW > 0} = {w I ^c-"'-^'V < 1}. 

But each summand of the sum in the right expression is an exponential of a linear function of w, hence 
convex. A sum of convex functions is convex, and its level set is a convex set. □ 

Corollary 1. We can find a pair (c, w), where c = infw c„, w ■ t = I, and w is a limit point of a sequence wi 
with Cwg converging to c, as a solution to a quasiconvex program. 

Proof. We form nested convex families from the closures of the level sets of the functions qi{w) = c^,j. The 
result follows from the definition of c^^■ = maxcvv. □ 

Theorem 1. If (c, w) is found as the optimal solution to the quasiconvex program defined as above for a 
recurrence for F (x) with target vector t, thenf{n) = F{nt) = 0{c"). 

Proof. If Cw,i = c < +CX3 for all terms /, the result follows from Lemma|3] And if some term has w ■ 6ij < 0, 
or has more than one summands, then Cwj = +oo but any sequence wi converging to w has Cw^ converging 
to +00, so the the upper bound is infinite and the result is vacuously true. The only remaining case is that 
some terms have a single summand J, with w ■ Si = 0. Such terms can not contribute to asymptotic growth 
of F{n t) and the result follows by applying Lemma|3]to the recurrence formed by omitting them from the 
definition of F. □ 

Therefore, we can use quasiconvex programming to find the weight vector w yielding the tightest pos- 
sible upper bound 0{c") on the asymptotic behavior of our recurrences. Further, by finding a basis for the 
quasiconvex program, we can determine the recurrence terms that are critical for its asymptotics. The re- 
striction w -t = \ does not affect quasiconvexity, and in fact aids in the solution of our problem by reducing 
the effective dimension of the quasiconvex program to J — 1 . 

5 Implementation 

In this section we discuss two implementations of our optimization algorithms. Our implementations use 
a numerical improvement technique which we call smooth quasiconvex programming and which will also 
feature in our later lower bound proof. 




Fig. 1. Example showing the difficulty of applying standard gradient descent methods to quasiconvex pro- 
gramming. The function to be minimized is the maximum distance to any point; only points within the 
narrow shaded intersection of circles have function values smaller than the value at point w. 



As stated earlier, generalized linear programming techniques can be used to solve our quasiconvex pro- 
grams in 0{t) steps, where r denotes the number of terms and each step consists of solving a constant 
sized subproblem. Such a subproblem has only a constant number of potential bases, and we can test a 
basis by solving an algebraic system of equations in the variables c'^'W where w[k] denotes the kth coordi- 
nate of w. However this approach appears likely to be cumbersome in practice. Instead, we implemented a 
hill-climbing scheme for finding numerically the optimal value (c, w) for our quasiconvex program and for 
certain other quasiconvex programs. 

We have in mind two different purposes for an implementation of our analysis algorithm. First, such an 
implementation could be used for exploratory analysis: computing a rough estimate of the running time of 
an algorithm, with enough accuracy to determine whether it improves other similar algorithms for the same 
problem, and determining the worst cases in an algorithm's case analysis, in order to refine that analysis to 
produce a better algorithm. For this sort of application, it is important that the running time be fast enough 
to be usable at interactive or near-interactive speeds; an implementation using floating point arithmetic is 
appropriate. Second, we would like to be able to publish guaranteed worst-case bounds on algorithms, for 
which approximate numeric schemes such as floating point that lack error bounds are inappropriate; in this 
setting the longer running times associated with exact or interval arithmetic may be acceptable. 

With these considerations in mind, we implemented our algorithm twice, once using floating point and 
a second time using XR |6l, an exact real-number computation package for the Python programming lan- 
guage (W]. The results of the second implementation are guaranteed to be valid upper bounds for the recur- 
rence in question. In XR, a real number p is represented as a data structure that is capable of constructing 
a multiprecision integer representing the rounded value of 2^p, for any integer b; this rounded value is re- 
quired to be within unit distance of the true value. We extended this package so that it could evaluate as 
exact real numbers the values Cw given an input vector w the coordinates of which are also real numbers, 
by performing an appropriate binary search using the values of the function r^ic). We then implemented 
a quasiconvex programming algorithm to search for a sequence of vectors W£ converging to the infimum 
(c, w) of the quasiconvex program. We describe in more detail below our exact arithmetic implementation; 
our floating point implementation is similar. 

If all functions qi{w) are quasiconvex, the function q{w) = max,^,(w) is itself quasiconvex, so we 
can apply hill-climbing procedures to find its infimum. However, although in our application the individual 
functions qj are smooth, their maximum q may not be smooth, so it is difficult to apply standard gradient 
descent techniques. The difficulty may be seen, for instance, in a simpler quasiconvex programming prob- 



lem: determining the circumradius of a planar point set (Figure Q. A basis for the circumradius problem 
may consist of either two or three points; the quasiconvex functions used to solve the problem are simply 
the distances (or squared distances) from each input point, and the function q{w) measures the distance from 
w to the farthest point. But if a point set has only two points in its basis, and our hill climbing procedure for 
circumradius has reached a point w equidistant from these two points and near but not on their midpoint, 
then improvements to the function value q{w) may be found only by moving w in a narrow range of direc- 
tions towards the midpoint. Standard gradient descent algorithms may have a difficult time finding such an 
improvement direction. Similar behavior arises naturally in our recurrence analysis problem. For instance 
in a recurrence in 1 12] for the time bound of a TSP algorithm on cubic graphs, there are only two critical 
terms despite the problem being defined over a three-dimensional vector space, and these two terms lead to 
behavior very similar to the circumradius example. 

To avoid these difficulties, we use the following algorithm, which we call smooth quasiconvex program- 
ming. We assume we are given as input a set of quasiconvex real functions qi{w). Further, we assume that 
for each / we also can compute a vector-valued function q*{w), satisfying the following properties: 

1. If qi{x) < qi{w), then {x — w) ■ q*{w) > 0, and 

2. If q*{w) • V > 0, then for all sufficiently small e > 0, qi(w + ev) < qi{w). 

For the circumradius example, for instance, should be a vector pointing from w towards the ith input 
point. The requirements on can be described geometrically, as follows: we assume that the level set qf^ 
is a smooth convex set, one that has at each of its boundary points a unique tangent plane. The vector qj (x) 

is then an inward-pointing normal vector to the tangent plane to qf''^^^ ntx (essentially it is just the negation 
of the gradient of q). The functions qi{w) arising in our recurrence analysis problem have this smoothness 
property, and (as we discuss in more detail in the next section) the vectors q*{w) can be constructed by 
evaluating the partial derivatives for each of the coordinates of x in the expression r^^i{cw) atx = w. 

Our smooth quasiconvex programming algorithm then consists of selecting an initial value for w, and a 
desired output tolerance, and repeating the following steps: 

1. Compute the set of vectors qj (w), for each / such that qi{w) is within the desired tolerance of max,- qi{w). 

2. Find a vector v such that v • q*(w) > for each vector qj{w) in the computed set. If no such vector 
exists, q{w) is within the tolerance of its optimal value and the algorithm terminates. 

3. Search for a value e for which q{w + ev) < q{w), and replace w by w + ev. 

Our actual implementation augments this procedure by an outer scaling loop that gradually decreases 
the tolerance, so that multiple terms of the recurrence can influence the computation in step 1 even when 
the current value of w is only a rough approximation to the true optimum. We also terminate the loop when 
the improvement to q{w) becomes much smaller than the tolerance, even when the termination condition of 
step 2 is not met, in order to handle situations in which the optimal basis is less than full-dimensional. 

The search for a vector v in step 2 can be expressed as a linear program. However, when the dimension 
of the quasiconvex program is at most two (equivalently, the number of variables in the recurrence to be 
solved is at most three) it can be solved more simply by sorting the vectors q* (w) radially around the origin 
and choosing v to be the average of two extreme vectors. 

In our implementation of step 3, we perform a doubling search for the largest e leading to a smaller value 
of q{w + ev), and then reduce the resulting e by a factor of two before replacing w, to attempt to control 
situations where the value w oscillates around the true optimal value. 

Both due to our use of exact real arithmetic, and due to the implementation in Python, a relatively 
slow interpreted language, our exact arithmetic implementation is not fast, taking several hours on a laptop 
computer to solve moderately sized 3-variable recurrences to 64 bits of precision. However our floating 
point implementation is able to run at interactive speeds, taking roughly one or two seconds on a recent- 
model laptop to solve recurrences such as the one in Table ^ We believe that significant improvements in 




Fig. 2. A portion of the infinite graph Gj* for the recurrence T{n,k) for listing maximal independent sets of 
cardinality k in an «- vertex graph, described in the introduction. The horizontal position of a vertex indicates 
its n coordinate and the vertical position indicates its k coordinate. To simplify the drawing, edges are shown 
in a confluent style ||9l in which multiple edges are allowed to merge before reaching their destination. Each 
vertex is labeled with the number of paths in the graph from that vertex to the origin. 

runtime of our exact arithmetic implementation would be possible both by tuning the implementation and 
by using a faster software base. However it is encouraging that, in the trials we attempted, the algorithm 
appears to exhibit linear convergence to the correct function value as well as to the optimal weight vector 
coordinates: the number of iterations of the algorithm appears to be proportional to the number of bits of 
precision desired. 

6 Lower Bounds 

In this section we prove that the upper bounds found by our optimal weighting technique are tight to within 
a polynomial factor 

In order to find lower bounds for the asymptotic behavior of our recurrences, it is useful to have the 
following combinatorial interpretation of their values. For any x G Z^, let i*{x) = argmaXjJ2jF{x — 6ij). 
That is, /* is the index of the term in the recurrence that determines the value of F{x). If fi{x) : Z*^ i— > Z is 
any function mapping vectors to recurrence term indices, define an infinite graph G^, the vertices of which 
are vectors in Z"^, with an edge from x to x — 6jj whenever / = /i(x) and djj is a summand in term / of the 
recurrence. Let '/r^(x) denote the number of paths from x to zero in G^. It is not hard to show by induction 
that vr,* (x) = F{x), so F can be interpreted as counting paths in a graph. Moreover, for any fj,, vr^(x) < F{x). 
An example of the graph G,* , for the recurrence discussed in the introduction, is shown in Figure |2l each 
vertex x in the figure is labeled with the number 7r,*(x) = F{x). We will find a lower bound for F{x) by 
counting paths in a graph G^, where ^(x) will not necessarily equal i*{x). 

Let X*{S) = (c, w) be the optimal solution to the quasiconvex program formulated earlier for the upper 
bounds for our recurrence, and let B be a basis of the quasiconvex program; that is, a minimal subset of 
terms of the recurrence such that the quasiconvex program formed from the subset has the same value 
X*{B) = X*{S) as that for the whole recurrence. For any term / in B, and any summand j in term /, let 
Pi J = c-^'-^'J, and let A = EPiJ^iJ- 



Lemma 5. For any term i in B, J2j Pij = 1- 



Proof. If term / has only one summand, then it belongs to B exactly when w ■ 5ij = and pjj = 1. Otherwise, 
qi{w) is continuous and term / can belong to B only when qi{w) = c. But then r^ic) = I — J2 Pi J = since 
qj is defined as having the value that makes this equation true, so J2 Pi J = 0. □ 

Lemma 6. The vector Di defined above is a scalar multiple of the vector q* defined for the smooth quasi- 
convex programming algorithm of the previous section. 

Proof. The vector q^w) is the vector such that ^,(w + ev) < w,(w) (for sufficiently small e) if and only 
if V ■ q* > 0. Expanding the definition of qj, as the value c such that r^^i^c) = 0, we see that q*{w) 
can equivalently be defined as a vector v such that r^^,^^yJ{c) > (for sufficiently small e) if and only if 
V ■ q* > O.Therefore, q* can be computed as the gradient of the vector function 0(;c) = rx^i{c), evaluated at 
X = w. Expanding this gradient by computing partial derivatives for each component of its vector argument, 
we arrive at the definition of D,. □ 

Lemma 7. There exist values bi, < bi < 1, so that J2^i — ^ '^^^ ^^'^^ S ^i^i <^ scalar multiple of 
the target vector t. 

Proof. The smooth quasiconvex programming algorithm of the previous section will find an improvement 
to solution (c, w) whenever there exists v, perpendicular to t, having positive dot product with all the vectors 
q*{w). The pair (c,w) used to define bj and D,- is assumed optimal, so can not be improved. Therefore, v 
does not exist and the origin must be contained in the convex hull of projections perpendicular to t of the 
vectors q*. Any vector in the convex hull of a set of vectors can be expressed as a convex combination of 
those vectors, and the same convex combination (when viewed in M.'^ rather than its projection perpendicular 
to t) has the property stated in the lemma. □ 

We are now ready to describe the graph used in our lower bound construction. For each x G Z^, we 
choose ii{x) to be one of the terms in the basis B, independently at random, with probability bf for choosing 
term /. We then let be the infinite graph formed from /i as described at the start of the section. For each 
choice of /i the number of paths vr^ {n t) from nt to the origin in G^ forms a valid lower bound for the 
quantity f{n) = F{n t) that we are trying to estimate, so the expected number of paths (averaged over the 
choice of /i) also forms a valid lower bound. 

To count the expected number of paths in G^, we use the following random walk process. Start from the 
vertex n t, and then from any vertex x choose randomly among the edges leading away from x, independently 
for each x. The set of edges at x is in one-to-one correspondence with the summands of term ij.{x), and we 
choose summand 7 with probability p^{x)j- As shown in Lemma |5] these probabilities add to one at each 
vertex. We continue this random walk process until we reach a vertex x with w ■ x = 0. 

Lemma S. If a path from nt to the origin can be formed by the random walk described above, it has 
probability c~" of being chosen. 

Proof. More generally it is easy to show by induction on the length of the path that a path from x to 3^ has 
probability c^'(y~^) of being chosen. The result follows from the choice of starting point x = nt and the 
constraint that w ■ t = I. □ 

Lemma 9. The random walk described above reaches the origin with probability il{n^^~^^^'^). 

Proof. By LemmaQ the projections perpendicular to t of the vertices of the path form an unbiased random 
walk in M^~^, with 0{n) steps of constant size, and the result follows from the standard theory of such 
walks. □ 



Theorem 2. /(«) = F{nt) = n{c"n^^-'^^/^). 



Proof. By Lemmas [8l and |9l there is the given expected number of paths in from nt to the origin. The 
result follows from the facts that this number of paths is less than the number of paths between the same two 
vertices in G/* (since /* is defined to maximize the number of paths) and that the number of paths in G,* is 
exactly □ 

7 Conclusions and Open Problems 

We have shown that the solution to the recurrence 

F{x) = max ^ F{x - 6ij), 
j 

for X = nt may be upper and lower bounded within a polynomial factor of c", where the number c can be 
computed as the solution to a quasiconvex program defined from the recurrence and the target vector t. 

It would be of interest to tighten these bounds: under what conditions can we determine the correct poly- 
nomial adjustment to the bound c"? It is consistent with our observations so far that F{n t) = (9(c"?i^l^'~^^/^) 
where |S| is the cardinality of a basis for the quasiconvex program. For instance this would fit the central 
binomial coefficients, with |B[ = 1 and J = 2, as well as the recurrence used as an example at the start of 
this paper with \B\ = d = 2. However there is too little evidence yet to state such a formula as a conjecture. 

More generally, the work here is only a first step towards the automation of backtracking algorithm de- 
sign and analysis. Perhaps it would also be possible to automatically perform some of the case analysis used 
to design backtracking algorithms, and to determine the appropriate variables to use in setting up the recur- 
rences used to analyze those algorithms, before automatically solving those recurrences, at least for simple 
constraint satisfaction type problems. It would also be of interest to find ways of specifying algorithms of 
this type in such a way that their correctness can be proven automatically, especially in situations where re- 
peated refinement based on our analysis tools has led to highly complex case analysis such as that appearing 
in Table [2 Also, while we can find tight worst-case bounds on the solution of the recurrence derived from 
an algorithm, it may not always be possible to construct an instance causing the algorithm itself to have 
that worst case time bound; it would be useful to determine conditions under which this recurrence-based 
analysis is tight. 

In another direction, the proof of Theorem |2l hints at a theory of duality for quasiconvex programs that 
it would be of interest to explore. 
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